Data Analysis and Visualization 

Introduction to Virtual Reality and Immersive Visualization 

Benjamin Weyers 
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One Minute Paper - Positive Feedback 



• Good Examples lllllll (7) 

• Clear English Mil (4) 

• Good slides Mil (4) 

• Humorous II (always tries (!) to make jokes) (2) 

• Man kann alle Inhalte verstehen II (2) 

• Really nice lecture II (2) 

• Sum up at the end of each part 

• Bester Vortrag seit 2 Jahren studieren, hat mich voll motiviert das Fach zu rocken. Freue 
mich auf Dezember © 

• Folien sind sehr ubersichtlich 

• Interessante Vorlesung mit Selbsexperiment 

• Gutes Tempo 

• Good structure 

• Viele Bilder 

• Sehr interessant und anschaulich 

• Science Applications 
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One Minute Paper -Negative Feedback 



• Show real application examples (Google Cardboard, 3D Rendering software) III (3) 

• Somewhat quiet III (3) -> Microphone 

• Voice modulation helps to get human attention II (2) 

• Nicht immer klar was wichtig / klausurrelevant ist II (2) 

• Inhalte veraltet II (2) 

• Nothing II (2) 

• More programming exercise -> Next Exercises on Web-based InfoVis 

• Include some assignment / project for rendering 

• Formeln werden nicht erklart 

• Use MatLab to show operations 

• Not going to deeply into topics 

• Mathematical Notation: v*M bad, M*v good 

• I didn't get the concept of the first two lectures 

• Folien ubersprungen 

• Luminance and Brightness: Difference? 

• Too long 

• Vague objective of the course 
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Closer look to (Visual) Analysis in the Simulation Loop 

L 
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Introduction - Topics 



• What is Virtual Reality? 

• VR Displays and Systems 

• Stereoscopic, Viewer-Centered Projections 

• VR in Scientific Visualization 
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Why Data Analysis & Visualization? 




JUQUEEN IBM Blue Gene/Q System, JSC, 
Forschungszentrum Julich GmbH 
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raw data 

Gigabyte-Terabyte-Petabyte 



• Generating insight from data requires data analysis. 

Hamming: „The purpose of Computing is insight, not numbers!" 
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Why Data Analysis & Visualization? 




JUQUEEN IBM Blue Gene/Q System, JSC, 
Forschungszentrum Julich GmbH 
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raw data 



Gigabyte-Terabyte- Petabyte Visualization 



• Generating insight from data requires data analysis. 

Hamming: „The purpose of Computing is insight, not numbers!" 



• Visualization is (primarily) being used for the analysis process 
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Why Data Analysis & Visualization? 
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1 



1 



010010101001010 

101001010101010 

10101110101011 

010111100 . . . 



raw data 
Gigabyte-Terabyte-Petabyte 



JUQUEEN IBM Blue Gene/Q System, JSC, 
Forschungszentrum Julich GmbH 



• Generating insight from data requires data analysis. 

Hamming: „The purpose of Computing is insight, not numbers!" 



• Visualization is (primarily) being used for the analysis process 

• Amount of raw data is rapidly increasing: Finer grids, 3-D, time-variant 

• Explorative versus confirmative analysis, Virtual Reality 
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Introduction - Topics 



• What is Virtual Reality? 

• VR Displays and Systems 

• Stereoscopic, Viewer-Centered Projections 

• VR in Scientific Visualization 
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Virtual Reality - A Definition 

L 



Virtual Reality (VR) is a computer generated 
environment. A user can interact with this 
environment and experience it with her 

natural senses. 
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What is Virtual Reality about? 






IMMERSION 












Navigation 

Manipulation 




INTERACTION 



...in real-time! 






3-D & multimodal 

• visual 

• acoustic 

• haptic/tactile 

• proprioceptive 
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3D Vision: Physiological & Psychological Cues 



Traditional CG: 

• Psychological clues 

- Perspective shortening 

- Occlusion of objects 

- Light and shadows 

- Texture gradients 
Atmospheric perspective 
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Perspective 




Drawing: Goldstein (WWW) 
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“Hi 




The Ames Room 



Pictures: Goldstein, Levine & Shefner (WWW) 




https://upload.wikimedia.org/wikipedia/commons/l/ll/Ames room.ogv 
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Occlusion 
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Pictures: Hubner (WWW) 





Light & Shadows 



Pictures: Levine & Shefner (WWW) 
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3D Vision: Physiological & Psychological Cues 



Traditional CG: 

• Psychological clues 

- Perspective shortening 

- Occlusion of objects 

- Light and shadows 

- Texture gradients 
Atmospheric perspective 




Virtual Reality: 

• Physiological clues 

- Stereopsis 

- Ocular motor factors 

■ Accommodation 

■ Convergence 
Motion parallax 
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Convergence 




B1 

Position 1 
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Drawing: Goldstein (WWW) 



A2 
B2 

Position 2 




“Hi 




Accommodation 



http://www.denstoredanske.dk 
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Motion Parallax 
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Tom Vaughan, 
www.cyberlink.com 





Motion Parallax 
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“Hi 




Stereopsis 



• Interocular distance (about 6 cm) 

• Disparity of images projected onto the retina 

• Processing in the visual cortex of the brain 

• Works for distances up to 7 m 



http://www.more3d.com/3-D/Stereoskopie.html 






Depth Cue 


Scope 
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Position Estimation 


Occlusion 


full range 


mono-ocular 


relative 
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bi-ocular 


relative 


Convergence 


< 2m 


bi-ocular 
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Accomodation 


< 2m 


mono-ocular 


absolut 


Image Blur 


full range 


mono-ocular 


relative 


Linear Perspective 


full range 


mono-ocular 


absolut 


Texture Gradient 


full range 


mono-ocular 


relative 


Relative Size 


full range 


mono-ocular 


absolute 


Known Size 


full range 


mono-ocular 


absolute 


Height in Visual Field 


> 30m 


mono-ocular 


relative 


Atmospheric 

Perspective 


> 30m 


mono-ocular 


relative 


Shape from Shading 


full range 


mono-ocular 


relative 


Shadows 


full range 


mono-ocluar 


relative 


Motion Parallax 


> 20m 


dynamic 


relative 


Accretion 


full range 


dynamic 


relative 
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Introduction - Topics 



• What is Virtual Reality? 

• VR Displays and Systems 

• Stereoscopic, Viewer-Centered Projections (VCP) 

• VR in Scientific Visualization 
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The Head Mounted Display 





CAE 




I. Sutherland 



Pictures: diverse Web Sites 
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Sony 



BOOM 






Retina Display, 
Microvision 



* j 










m 



Head-Mounted Displays versus Stereo Glasses 



Until 1994: 

VR = HMD + Instrumented Glove Stereo Glasses (here: Shutter) 










■ 



Stereo Parallax in Room-Mounted Displays 
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Tom Vaughan, 
www.cyberlink.com 




Stereograms 




zero parallax 




non-crossed 
(positive) parallax 
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crossed 

(negative) parallax 





■ 



Stereo Glasses: Shutter (“Active Stereo”) 



• Time Multiplex 

• Genlocking of multiple graphics hardware 




Infrared Stereo Emitter: for opto-electronical tracking 

Synchronization of glasses and graphics hardware 
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Stereo Glasses: Shutter (“Active Stereo”) 



16ms 16ms 16ms 16ms 



time 







displayed on the 
screen. 

The images switch 
so fast that without 
glasses the images 
seem to be blended 



projector 1 : displays image for left eye alternating with a black image 




projector 2: displays image for right eye alternating with a black image 




glasses: are opaque for one eye at a time, controlled by an infrared 
shutter signal that is synchronized with the projectors 
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Stereo Glasses: Polarization (“Passive Stereo”) 



• Linear or circular polarization 



• Swap (and Frame) locking of multiple graphics hardware 



• screen material may not destroy polarization (hot spot) 





Distortions in Static Stereograms 
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Adaptation of projection to the viewpoint 



projection plane 




projection plane! 




eye position 2 
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Electromagnetic Tracking - Polhemus 
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"mm 






Electromagnetic Tracking in Military 




]SF Simulator Courtesy of Boeing 
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“Hi 






Electromagnetic Tracking - Pros & Cons 



+ Position & orientation 
+ No occlusion 

- sensitive to ferromagnetic materials in the environment 

- Accuracy decreases with growing distance between sender and 
receiver 

- Rather large sensors (1-2 cm) 
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Optoelectronic Tracking - A.R.T. 



Picture: A.R.T. GmbH 
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“Hi 








Optoelectronic Tracking - Pros & Cons 



+ Accuracy, latency sample rate (of today’s systems) 

+ Passive markers: No cabling, nearly non-intrusive 

- Occlusion 

- No orientation (combine 3 non-collinear markers) 

- Calibration process (rather comfortable in today’s systems) 
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The Principle of Viewer Centered Projection (VCP) 



stereo parallax motion parallax 
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Diagonal Projection 





Basic Operations 
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r 



Shearing of the View Volume 

L 



P — S ' H f^X Ey 

Pi - ~E^) 



/ 1 0 0 0 \ 

0 10 0 



Ez Ez 

0 0 0 1 / 
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The Projection Matrix 



Parameters: 

• Position of the view window: left, right, top, bottom 

• Near and far clipping plane: near, far 

• Eye position E 




o 



2e z 

o—u 

-2(e y -^) 



o—u 



q±u 

o—u 




0 

0 



2 e z -f-n 
-e z (2 e z -f-n) 



—n 

+2(f-e z )(n—e y 



f-n 
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Viewer Centric Projection (VCP) on a Workbench 





% 

Tracked Virtual Table 



- tillable BARON table 
- optically tracked 
- real-time recalibration 
- extended working volume^, 
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Effect of Motion Parallax 



Courtesy of Bill Sherman 
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3D Vision: Physiological & Psychological Cues 



Traditional CG: 

• Psychological clues 

- Perspective shortening 

- Occlusion of objects 

- Light and shadows 

- Texture gradients 
Atmospheric perspective 




Virtual Reality: 

• Physiological clues 

- Stereopsis 

- Ocular motor factors 

■ Accommodation 

■ Convergence 
Motion parallax 
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PowerWall 




H Prof. Dr. Torsten W. Kuhlen | Immersive Visualization | WS 2014/15 | 
Course on Virtual Reality - Name des Kapitels 







Stereo Glasses: Polarization (cont.) 






Horizontal 
■ Polarization 
Filter 



• • 



••• 



• • • i 



• * 



Permant 

Projection of both 
Views 

no left/right Eye 
flipping -> Passive 









Vertical 
■ Polarization 
Filter 



Polarized Glasses no 
Shutter -> Passive 
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Multi Projector PowerWall 
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2160 pixels 



High Resolution PowerWall @ AICES Graduate School 




4096 pixels ► 
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The CAVE, 1993 
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CAVE Installations in Germany in 2004 



n 

□ 

n 

“i 

n 










SCHLESWIG-HOLSTEIN 



MECKLENBURG-WESTERN 



Schwerin 



BERLIN 



Magdeburg 



Jre&deo 



THURINGIA 



WiEsbadEfl 



Wnyence 



,S3a'rijrucl[Eft 



BADEN- WURTFEMHERIj 



Front, back, left, right, bottom, top 
Front, left, right, bottom, top 
Front, back, left, right, bottom 
Front, left, right, bottom 
Front, left, bottom 
Front, left, right 



(2004, maybe incomplete) 



Research Institutes 

• Stuttgart, Fraunhofer IAO/IPA (6) 

• Bonn caesar (7) 

• Darmstadt, Fraunhofer IGD (5) 

• Magdeburg, Fraunhofer IFF (5) 

• Berlin Fraunhofer IPK (5) 

• Berlin Fraunhofer First (5) 

• St. Augustin Fraunhofer IMK (4) 

• Braunschweig DLR (3) 



Industry 

• Munchen, BMW (4) 

• Sindelfingen, DaimlerChrysler (5) 

• Ulm, DaimlerChrysler (4) 

• Wolfsburg, VW (3) 

University 

• Aachen RWTH, RZ (5) 

• Bielefeld Univ. AG Kl (3) 

• Chemnitz, TU (5) 

• Stuttgart Univ., HLRZ (4) 



H Prof. Dr. Torsten W. Kuhlen | Immersive Visualization | WS 2014/15 | 
Course on Virtual Reality - Name des Kapitels 





The aixCAVE @ RWTH Aachen 






I 



S»8« 



v I" 



0OAAHTWM3HOAA 
Wm ;I2H3VII 



Same visual perception in real & virtual world! 

Realize stereo AND motion parallax 




53 
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PC Clusters for Immersive Displays 






VR Cluster 



GPU 1 



CPU 4 



fjp 

GPU 4 



CAVE 



GPU 2 



GPU 3 
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The aixCAVE Specs 



• Footprint approx. 5.50 x 5.50 m, 3,30 m high 

• 24 WUXGA Active Stereo Projectors 




Approx. 3200 x 2000 pixels per wall 
Approx. 3600 x 3600 Pixels at the floor 
Brightness: Nearly 300.000 ANSI Lumen 
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VCP in the aixCAVE 





Topics 



• What is Virtual Reality? 

• VR Displays and Systems 

• Stereoscopic, Viewer-Centered Projections 

• VR in Scientific Visualization 
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Motors & Turbines 
Twin Extruders 
Nasal Airflow 
Bood Flow 
Crash Simulations 
Material Science 
Pig housing 




I A, VR Group I 



VR Group 



inner 

on Virtual Reality - Name des Kapitels 



IKV, V 









Video: Counter Propfan 
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L. Jerabkova 
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Motivation 





Increasing Computer Power 

Comples Simulations 
(3D, fine grids, unsteady 



JUQUEEN IBM Blue Gene/Q System, JSC, 
Forschungszentrum Julich GmbH 



SMP Cluster in Aachen 



Explorative (instead of confirmative) 
analysis in 3-D space 



• Trial & Error process 

• Interactivity! 

Challenges 

• Performance: Interactivity versus large datasets 

• HCI: How to create and interact with complex vis objects in 3D? 

How to navigate in space and time? 
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Levels of Interactivity 



Raw Data ► Vis. Primitives — ► Graphics Primitives ► Pixels 




• What is interactive? 

- Nothing 

■ Image, Animation (Movie) 

- Rendering only 

■ Navigation, selection & animation of pre-computed vis. primitives 

- Filtering, Mapping, Rendering 

■ Interactive Visualization, Interactive Post Processing, Explorative Analysis 

- Simulation, Filtering, Mapping, Rendering 

■ Interactive Simulation, Computational Steering, 

Interactive Supercomputing, Human-ln-the-Loop 
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Architecture for VR-based Scientific Visualization 



Simulation 



1 


ife 












lx 








Raw Data 



Dilemma: Intera 



cSj 9 ts 




Postprocessing: 
eature Extraction & 
Rendering 



us large datasets 
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Parallel Framework 



A. Gerndt, M. Wolter, B. Hentschel, I. Assenmacher, I. Tedjo, S. Pick, A. Kelle-Emden 



[IMACS 2000, SC 2004, EGVE 2003, EGPGV 2006, SC 2006, EGPGV 2007, ParCo 2007, SEARIS 2008, CGF 28(6)] 





Vista FlowLib 

Rendering 

m ■ ■ 

W ■ ■ ■^^■REALITY fof 

■■ ■ ■ ■■SCIENTIFIC 

■ ■mtECHNICAL 

■ ■ m applications 





Socket 










J 











TCP/IP 

K — — —J 





0 200 400 600 

Run Time [s] 
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1000 










Interactive Particle Tracing on the GPU 



M. Schirski 
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“The Virtual Windtunnel Revisited” 



Marc Schirski et al. 




GPU-based Real-Time Particle Tracing 
Works for structured as well as unstructur] 
Advanced Billboard Rendering (Virtual Tubelets) 



05p r lf!6Ci^riy^M/.2:CHil5 |Pi0i&iterve(v^tiSz2t9(( \ 2Q06, MMVR 20 
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-varying datasets 



E 2007] 







Handling Large Data: Regions of Interest 



C Back-End 

"1 






I 



Parameter 

Reception 



± \ 










1 






1 




1 








! 1 




Parallel ROI 
Extraction 



ROI 

Transmission 

I 



Visualization Front-End 

User Input / 

ROI Specification 






Interactive 

Exploration 




Prof. Dr. Torsten W. Kuhlen | Immersive Visualization | WS 2014/15 | 
Course on Virtual Reality - Name des Kapitels 










Framework for Interactive FlowVis & More 



Marc Wolter et al. 




© Ultra High Performance 
© Large Storage 
© High Latency 




rendering nodes 



© Medium/High Performance 
® Medium/Small Storage 
© Low Latency/Real-Time 



i 



Large-Scale 
Data Processing 
(Low- & Mid Frequency 



i 



Interactive Operations 
(High Frequency, 
Navigation) 



\ Xml ■ ftk. ■ f^i 14 re 

lbllt 



VIRTUAL 
REALITY for 
SCIENTIFIC 
TECHNICAL 
APPLICATIONS 



tasks) [IMACS 2000, SC 2004, EGVE 2003 , EGPGV 2006, SC 2006, EGPGV 2007, ParCo 2007, SEARIS 2008, CGF 28(6)] 
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VR-Systems 





Acceleration Sensor 


i 


Loud Speaker 


n 


• • • 




... hearing 






Optical Tracking 




Displays 






Depth Camera 


(O) 


visual system 






Microphone 




Motion Platform 






Camera(s) 


Human 


proprioception 






Pressure Sensors 




Force Feedback 






Mechanical Tracking 




haptic 
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Topics 



• What is Virtual Reality? 

• VR Displays and Systems 

• Stereoscopic, Viewer-Centered Projections 

• VR in Scientific Visualization 
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